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Abstract 

Background: Previous studies have suggested that the study condition of an item influences how the item is 
encoded. However, it is still unclear whether subsequent source memory effects are dependent upon stimulus 
content when the item and context are unitized. The present fMRI study investigated the effect of encoding activity 
sensitive to stimulus content in source memory via unitization. In the scanner, participants were instructed to 
integrate a study item, an object in either a word or a picture form, with perceptual context into a single image. 

Results: Subsequent source memory effects independent of stimulus content were identified in the left lateral 
frontal and parietal regions, bilateral fusiform areas, and the left perirhinal cortex extending to the anterior 
hippocampus. Content-dependent subsequent source memory effects were found only with words in the left 
medial frontal lobe, the ventral visual stream, and bilateral parahippocampal regions. Further, neural activity for 
source memory with words extensively overlapped with the region where pictures were preferentially processed 
than words, including the left mid-occipital cortex and the right parahippocampal cortex. 

Conclusions: These results indicate that words that were accurately remembered with correct contextual 
information were processed more like pictures mediated by integrated imagery operation, compared to words that 
were recognized with incorrect context. In contrast, such processing did not discriminate subsequent source 
memory with pictures. Taken together, these findings suggest that unitization supports source memory for both 
words and pictures and that the requirement of the study task interacts with the nature of stimulus content in 
unitized source encoding. 

Keywords: Source memory, Content, Study processing, fMRI, Unitization 



Background 

Neural correlates of successful memory formation have 
been investigated with the functional magnetic reson- 
ance imaging (fMRI) subsequent memory procedure, in 
which neural activity during encoding for later remem- 
bered items is compared with activity for later forgotten 
items. Subsequent memory effects are often expressed as 
enhanced encoding activity for remembered items in 
several brain regions, such as the left prefrontal cortex, 
the fusiform area, and the medial temporal lobe (MTL) 
[1-3]. Further, subsequent memory tends to reveal the 
influence of the study condition in which the item is ex- 
perienced at the cortical level. Seminal behavioral studies 
of memory have suggested that memory representation 
results from how an item is processed at encoding [4-6]. 
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From this perspective, episodic memory is the by- 
product of study processing based on the circumstances 
in which an event is experienced. In accordance with 
this theoretical principle, previous fMRI studies of sub- 
sequent memory effects have shown that cortical encod- 
ing activity for successful memory of an item depends 
upon the nature of the study task, the modality of the 
study item, and the material of the study item [2,7-9]. 

In these studies, the differences in study conditions, in- 
cluding study task (semantic vs. non-semantic) or stimulus 
content such as modality (visual vs. auditory) and material 
(verbal vs. pictorial), dissociated the cortical regions where 
neural activity for successful memory formation was iden- 
tified. For example, subsequent memory of words was pre- 
dicted in the left inferior prefrontal cortex and the left 
fusiform region, whereas encoding activity for pictures 
was identified in the right inferior prefrontal cortex and 
bilateral fusiform areas [2]. The MTL effects of memory 
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formation also showed left lateralized activation for words 
and right lateralized activation for pictures and faces [9]. 
Semantic encoding of successfully remembered words was 
related with activity in the left and medial prefrontal re- 
gions; however, non-semantic subsequent memory effects 
were found in the right prefrontal cortex, bilateral intra- 
parietal sulci and bilateral fusiform gyri [7]. Moreover, sub- 
sequent memory effects for visually presented words were 
reported in the right fusiform cortex, while auditory subse- 
quent memory effects were identified in bilateral superior 
temporal sulci [10]. 

However, relatively little is known about whether the 
formation of associations, as opposed to encoding of sin- 
gle items, also exhibits study condition-specific effects of 
encoding. The extant findings of task-dependent effects 
in memory associations are less consistent. For example, 
a study that compared task effects on associative memory 
of word pairs using semantic and phonological tasks found 
that extensive activity in the left inferior prefrontal cortex 
predicted word pairs that were later successfully remem- 
bered as an intact pair in both encoding tasks; task-specific 
effects, however, were negligible [11]. Another study that 
compared semantic versus perceptual study conditions for 
associative memory of word pairs reported that the same 
left inferior prefrontal regions were involved in subsequent 
associative memory effects common to both conditions; 
however, perceptual associative encoding effects were also 
found in the left temporo-occipital and bilateral parietal 
cortices as well as the right parahippocampal cortex [12]. 

Other studies of subsequent source memory (item-source 
associations) showed that neural correlates of source mem- 
ory differed depending upon study task or stimulus content 
[13,14]. Subsequent source memory of words studied with 
a semantic task was localized in the left temporo-occipital 
cortex, while successful source memory effects with a non- 
semantic encoding task were found in the right temporo- 
occipital region, revealing hemispheric differences between 
subsequent source memory effects that depend upon the 
study task [13]. Subsequent source memory effects also var- 
ied by stimulus content, such that source encoding activity 
for words were found in the left inferior prefrontal cortex, 
while successful source memory effects for objects were 
identified in the left perirhinal cortex [14]. 

In addition to these cortical effects, the MTL is known 
to be critical for forming associations [12,15,16]. From the 
memory process view, the MTL subregions are specialized 
for different memory processes. That is, the hippocampus 
and the parahippocampal cortex are integral to the forma- 
tion of associations based on recollection, whereas the 
perirhinal cortex tends to support familiarity-based item 
recognition [17-21]. Functional dissociation within the 
MTL for different memory processes has been supported 
by findings that increased activity in the hippocampus pre- 
dicts subsequent recognition based on recollection (e.g., 



source memory) but not familiarity [22-24], whereas 
familiarity-based judgments (e.g., item memory) have been 
related with increased perirhinal activity during encoding 
[18,22,23]. It was further proposed that the perirhinal cor- 
tex could support source memory based on familiarity 
through unitization of an item and context [25]. As op- 
posed to the conventional way of binding (i.e., non-unitized 
encoding), in which separate item and context are to be 
combined through associations, unitization is assumed to 
bind context to the item as an item feature. From this view, 
source memory through unitization may be supported by 
familiarity. Further, source memory via unitization was 
supported by perirhinal activity, which is consistent with 
the proposal of the perirhinal cortex for familiarity-based 
source recognition [25,26]. 

It has also been suggested that MTL subregions are in- 
volved in encoding of heterogeneous classes of stimulus 
content [22,27]. First, the hippocampus is involved in 
processing of content-independent binding. For instance, 
encoding activity in the anterior hippocampus was identi- 
fied for recollection of both objects and scenes, reflecting 
the content- independent role of the hippocampus [28]. 
Other MTL subregions selectively engage in processing of 
different types of content. Specifically, the parahippo- 
campal cortex is selective to visuo-spatial processing such 
as scene processing [29-33]. The perirhinal cortex is 
involved in object processing, and the role of perirhinal 
cortex in object encoding was found across different rec- 
ognition judgments (i.e., Remember, Know, and New) and 
source memory [29]. Similarly, a MTL subsequent mem- 
ory study that examined source encoding with objects and 
scenes reported the dissociation of the MTL cortex by 
content: the perirhinal cortex contributed to source en- 
coding for objects, but the parahippocampal cortex sup- 
ported source encoding of scenes [32] . 

Though previous studies suggested content-dependent 
encoding effects, most of those studies examined content- 
dependent effects with non-unitized encoding tasks or fo- 
cused on MTL activity. As alluded to earlier, unitization of 
an item and context may change the way that the item 
and context are bound; therefore, unitized source memory 
may be supported by different encoding activity at the cor- 
tical level. Considering that the memory process involved 
in unitized encoding may be different from the memory 
process for non-unitized encoding, an interesting question 
is whether subsequent memory effects specific to stimu- 
lus content would appear with unitized encoding both 
at the cortical and MTL levels. This is important, as it in- 
vestigates how encoding activity specific to each trial (i.e., 
stimulus content) is influenced by study processing across 
trials (i.e., encoding operation) in forming associations. 
Previous studies that used non-unitized encoding tasks 
reported content-independent effects in the hippocam- 
pus and content-dependent effects in the cortical areas. 
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However, it is not clear whether unitized encoding is sen- 
sitive to content-dependent subsequent memory effect. 
Additionally, given that non-unitized encoding tends to 
elicit hippocampal activity for source memory formation 
across different types of content, it is of interest whether 
unitized encoding would show similar MTL source mem- 
ory effects across verbal and pictorial content. In what 
follows, we report a study of the formation of source 
memory for words and pictures via unitization. 

Participants were presented a list of study items (i.e. 
words and pictures) on a colored background and were 
instructed to imagine the item in the background color. 
This meant that participants actively had to generate a 
mental presentation for words but not necessarily for pic- 
tures. Thus, imagery-based operations required by the study 
task would have different effects depending on whether the 
content was verbal or pictorial. By comparing encoding ac- 
tivity of words and pictures varied by later source memory 
performance (source correct vs. source incorrect), we inves- 
tigated the effect of imagery-based operations in source 
memory by stimulus content. 

Previous studies of subsequent memory effects have 
reported encoding activity for perceptual processing of 
visually-presented stimuli in the parietal and occipital corti- 
ces, as well as the fusiform cortex [10,13,34-36]. Further, 
perirhinal activity has been reported for the formation of 
source memory with objects through unitization [29,31,32]. 
Thus, it is likely that parietal/occipital activity as well as 
perirhinal activity would contribute to source encoding of 
both types of content. On the other hand, participants had 
to generate a visuo-spatial representation of the word and 
place it in the color context for words, whereas pictures 
were already presented in the necessary visual form. That 
said, visuo-spatial processing and binding would be critical 
for forming successful source memory with words by en- 
gaging in the study task, but less so with pictures. Then, 
neural activity for visuo-spatial processing and binding 
would distinguish words with contextual information from 
words without correct context to a greater extent than such 
activity with pictures, which would emerge as content- 
dependent subsequent source memory effects. Further, as 
pictures were presented in the pictorial form during study, 
the brain regions that showed picture-preferential process- 
ing (pictures > words) regardless of later source memory ef- 
fects would be the areas where subsequent source memory 
effects selective to words would be identified. 

As such, we hypothesized the following for the present 
experiment: 

(1) For subsequent source memory effects independent 
of stimulus content, unitization of item and color 
would be supported by encoding of visually 
presented objects, as evidenced by increased activity 
in the parieto-occipital cortices, the fusiform gyrus, 



and the perirhinal cortex, in addition to the 
established role of hippocampal activity in source 
memory formation overall. 

(2) Study requirement for engaging in visuo-spatial 
processing and binding for source memory 
formation would affect words more than pictures, 
and this would call for greater subsequent source 
memory effects selective to words in the ventral 
visual pathway and the parahippocampal cortex. 

(3) Subsequent source memory effects for words would 
indicate that these words were processed more like 
pictures, based on the overlap of regions where 
pictures were preferentially processed compared to 
words such as parietal and occipital regions. 

Results and discussion 

Behavioral results 

Proportions of recognition judgments (source correct, 
source incorrect, item miss) by stimulus content (word, 
picture) are displayed in Table 1. The proportions of cor- 
rect source judgments for both stimulus contents were 
significantly greater than the chance level (.25), £s [23] > 
15.82, ps < .001. There was no main effect of stimulus 
content, but there was a significant interaction between 
content and recognition judgment, F[2 46] = 14.42, p < .001. 
Follow-up tests showed that more source correct judg- 
ments were made with words than pictures, £[23] = 2.34, 
p < .05, whereas more source incorrect judgments were 
made with pictures than words, £[23] = 5.26, p < .001, al- 
though item misses were greater with words than pictures, 
£[23] =3.09, p < .005. Further, using d' as a measure of 
recognition accuracy, more accurate source judgments 
were made for words (1.73 [.15]) than for pictures (1.27 
[17]), £[23j =4.28, p< .001. 

Study reaction times are also shown in Table 1, segre- 
gated by stimulus content and later source memory judg- 
ment. A 2 x 2 ANOVA with stimulus content (word, 
picture) and source judgment (source correct, source incor- 
rect, item miss) as factors was conducted on these data. 
The ANOVA showed the main effect of stimulus content, 
indicating longer reaction times for pictures than for words, 
-F[l 23] = 8.59, p < .005. However, there was neither a main 



Table 1 Mean test response proportions (SEM) and mean 
study reaction times (in ms) as a function of stimulus 
content and memory judgment 



Stimulus 




Studied 




New 




content 


Source 


Source 


Miss 


Correct 


False 




correct 


incorrect 




rejection 


alarm 


Word 


.69 (.03) 
1 656 (84) 


.19 (.02) 
1671 (80) 


,12(,02) 
1661 (83) 


.91 (.03) 


.09 (.03) 


Picture 


.66 (.02) 
1 675 (84) 


.26 (.02) 
1 733 (83) 


.08 (.01) 
1 767 (74) 


.93 (.01) 


.07 (.01) 
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effect of source judgment nor an interaction between 
stimulus content and source judgment on study times. 

fMRI Results 

Analyses of subsequent source memory effects were 
based on contrasts between encoding activity of study 
items that were later accurately endorsed with the stud- 
ied color (source correct) as opposed to encoding activ- 
ity of study items that were later recognized but with an 
incorrect color (source incorrect). Item misses were not 
included in the analysis due to an insufficient number 
of trials. 

Neural activity for source memory formation independent 
of stimulus content 

First, we sought neural activity associated with the for- 
mation of source memory independent of stimulus content 
via unitization. In order to find subsequent source memory 
effects, the statistical parametric mappings (SPMs) for the 
main effect of source memory across stimulus content 
(source correct > source incorrect) were exclusively masked 
with the SPMs for the interaction of source memory by 
content. The outcome of this analysis is shown in Table 2. 
Source encoding activity was localized exclusively in the 
left hemisphere, including superior to inferior lateral parts 
of the prefrontal cortex, the orbito-frontal region, the lat- 
eral parietal and occipital cortices, and the fusiform gyrus 
(Figure 1A). Source memory independent of stimulus con- 
tent was also identified in the left perirhinal cortex ex- 
tending to the left hippocampus (Figure IB). The mean 
parameter estimates extracted from all of these clusters 
demonstrated that encoding activity was greater for source 
correct than for source incorrect items, and these dif- 
ferences were independently reliable for both stimulus 
content from all clusters (ps < .05). The contrast in the 
opposite direction (source incorrect > source correct ex- 
clusively masked with the interaction) did not reveal any 
suprathreshold clusters. 



Neural activity for source memory formation depending 
on stimulus content 

Next, we looked for encoding activity for source memory 
varied by the content of the stimulus through integrated 
encoding of item and context. Subsequent source memory 
effects dependent on content were identified by exclusively 
masking source memory effects in one stimulus content 
type (e.g., source correct > source incorrect with word tri- 
als and vice versa) with subsequent source memory effects 
in the alternate content type (e.g., source correct > source 
incorrect with picture trials and vice versa). This procedure 
revealed only subsequent source memory effects selective 
to words, which occurred in the superior medial parts of 
the left frontal lobe, the right insula, and the left ventral 
visual stream (Table 3). As illustrated in Figure 2, MTL 
source memory effects selective to words were identified in 
bilateral middle/posterior parahippocampal cortices, and 
activity in the right parahippocampal cortex extended to 
the vicinity of the right hippocampus. ANOVAs on param- 
eter estimates extracted from all clusters showing word- 
dependent source memory effects revealed significant 
interactions between stimulus content and source memory 
judgment, Fs[l 23] > 5.8, ps < .05. Follow-up tests revealed 
that words that were subsequently remembered with cor- 
rect source information showed greater encoding activity 
in these regions than words recognized with incorrect 
source information (ps < .001), whereas pictures did not 
differ by source memory in these regions. Subsequent 
source memory effects selective to pictures did not reveal a 
single cluster over the threshold. 

Overlap between subsequent source memory effects and 
processing effects of stimulus content 

We also examined whether subsequent source memory 
effects depending on content overlapped with brain re- 
gions where corresponding stimulus content trials elicited 
more activity than alternate content trials. Subsequent 
source memory effects for one content type (e.g., source 
correct > source incorrect for words and vice versa) were 



Table 2 Subsequent source memory effects independent of stimulus content 



Coordinates (x y z) Z (# of voxels) Region BA 



Non-MTL effects 














-12 


48 


42 


4.42 


104 


L superior frontal gyrus 


8/9 


-42 


9 


36 


4.30 


56 


L inferior frontal gyrus 


44 


-9 


9 


-21 


4.25 


39 


L inferior orbito-frontal cortex 


47 


-42 


-81 


33 


4.47 


156 


L angular gyrus/lnferior/superior parietal 
lobule/ Mid occipital cortex 


19/39 


-54 


-57 


-21 


3.52 


39 


L fusiform gyrus 


37 


MTL effects 














-21 


-15 


-24 


3.84 


39 


L perirhinal cortex/anterior hippocampus 
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Figure 1 Content-independent subsequent source memory effects. A: Lateral content-independent subsequent source memory effects. 
B: MTL content-independent effects overlaid on mean anatomical image across subjects. Mean parameter estimates elicited by stimulus content 
(sc: source correct, si: source incorrect, ps < .05). 



inclusively masked with corresponding effects for content 
processing (e.g., words > pictures and vice versa). No over- 
lap was found either with word or picture source memory 
effects in the area where words had elicited more activ- 
ity than pictures. Rather, subsequent source memory ef- 
fects for both words and pictures did overlap with brain 
regions where pictures had elicited more activity than 
words (Figure 3). Source encoding activity for pictures 
was overlapped with picture-preferential processing areas 
in the bilateral fusiform areas and the left superior parietal 
lobe. Extensive overlap was found between word source 
encoding activity and picture processing in bilateral para- 
hippocampal cortices and occipital regions. 

Discussion 

The present study investigated content-dependent source 
memory effects by means of an integrated imagery oper- 
ation. Successful source memory that did not differ between 



the two stimulus content types was predicted in the left in- 
ferior frontal gyrus (LIFG), the lateral parietal/occipital 
lobes, and the fusiform gyrus. In addition, MTL subsequent 
memory effects independent of stimulus content were 
identified in the left perirhinal cortex extending to the left 
anterior hippocampus. The formation of source memory 
selective to words recruited activity in the left frontal cortex, 
the ventral visual stream, and the bilateral parahippocam- 
pal cortices extending to the vicinity of the right poste- 
rior hippocampus. Picture-dependent subsequent source 
memory effects did not emerge over threshold. Finally, sub- 
sequent source memory effects for words extensively over- 
lapped with the areas where pictures were preferentially 
processed. 

Content-independent subsequent source memory ef- 
fects were identified in a number of cortical regions. 
First, subsequent source memory effects were evident in 
the LIFG extending to the superior frontal regions. LIFG 



Table 3 Subsequent source memory effects selective to words 



Coordinates (x y z) 




Z (# of voxels) 




Region 


BA 


Non-MTL effects 














-15 


36 


30 


3.59 


27 


L medial frontal gyrus 


32 


-24 


27 


54 


4.25 


5/ 


L middle frontal gyrus 


8 


27 


3 


24 


4.06 


40 


R insula/caudate 




-3 


-3 


15 


3.64 


---11 


Thalamus 




-42 


-57 


-18 


3.86 


49 


L fusiform gyrus 


37 


-42 


-69 


9 


4.27 


// 


L middle occipital gyrus 


39 


MTL effects 














-33 


-27 


-21 


3.61 


1/ 


L parahippocampal cortex 


36 


27 


-33 


-12 


4.39 


5/ 


R parahippocampus/ hippocampus 
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Figure 2 Word-dependent subsequent source memory effects in the MTL. Effects are overlaid on a section (y = -33) of the across-subject 
mean anatomical image. 



effects have been found in the formation of associative 
memory [11,12,37,38] and source memory [14,34] with 
both semantic and non-semantic encoding tasks. The 
current finding of LIFG involvement in integrated source 
encoding is consistent with the proposal that the LIFG 
is crucial for the formation of associations regardless of 
the type of study processing [11,38]. Encoding activity 
for source memory independent of stimulus content was 
also found in the left lateral parietal cortex, spanning 
from the angular gyrus to the superior parietal/occipital 
regions. These regions, especially the intra-parietal sul- 
cus, are known to engage in goal-directed attentional 
processing [39,40], multi-modal processing [35,41], and 
perceptual processing [35,36,42], in addition to encoding 
of both item and source memories [7,10,18,43]. Previous 
studies also reported subsequent source memory effects 
in the LIFG and the left parietal regions with non- 
unitized encoding tasks. Thus, the present findings of 
the LIFG and the left parietal activity through integrated 
binding of item and context indicates that neural activity in 
these regions supports the formation of successful source 
memory mediated by perceptual processing through unit- 
ized encoding as well as non-unitized encoding. 




Figure 3 Overlap between content-dependent subsequent 
source memory effects and content-preferential processing 
areas. A: Cortical regions where subsequent source memory effects 
elicited by words (blue) and pictures (red) overlap with regions 
where activity is greater on picture trials than word trials 
(thresholded at p < .001, yellow). Areas of overlap between words 
and pictures are shown in purple. B: Overlap between word-dependent 
source memory effects and picture processing in bilateral 
parahippocampal cortices. 



In parallel with these cortical effects, content- 
independent subsequent source memory effects were 
identified in the left perirhinal cortex extending to the 
left anterior hippocampus, which is consistent with pre- 
vious reports of perirhinal activity for source memory 
formation with objects [14,29,33] and prior findings of 
hippocampal activity for source encoding in general 
[12,31,32]. The formation of associations via unitization 
of an item and its contextual details is known to be 
supported by perirhinal activity, as the unitized repre- 
sentation of the item with context as an object can be 
recognized based on familiarity [44,45]. The current find- 
ing of perirhinal activity in source encoding with both 
words and pictures extends previous findings of perirhinal 
involvement in associations [18,19,32,33,46] in that source 
memory for both verbal and pictorial content is supported 
by perirhinal activity through unitization. Further, activity 
in the hippocampus has been found in subsequent source 
memory in a content-general manner, which confirms the 
established role of the hippocampus in memory associa- 
tions overall [19,28,31]. 

While investigating content-dependent subsequent 
source memory effects, we found only word-dependent 
effects. These effects were evident in the visual ventral 
stream, the region critical for object identification (i.e., 
the "what" pathway), which suggests that accurate source 
memory of words was mediated by visuo-spatial object 
processing more so than inaccurate source memory of 
words. That is, words that were subsequently remembered 
with source information tended to be the ones with more 
object-like processing during encoding, whereas words 
that were not accompanied with correct source informa- 
tion tended to be the ones that were not visualized well 
enough for imagery operation to occur at a level compar- 
able to those that elicited source correct judgments. a 
However, the level of object processing did not distinguish 
subsequent memory of pictures between source correct 
versus source incorrect. Considering that pictures were 
presented in the visual object form during encoding, ob- 
ject processing would likely occur for all pictures regard- 
less of whether they were accompanied with correct 
contextual information or not, hence the smaller impact 
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of object processing in subsequent source memory effects 
with pictures. This interpretation is supported by the sig- 
nificant interaction between stimulus content and source 
judgment revealing the differences in parameter estimates 
only with source correct words versus source incorrect 
words (see Results). In addition, insular activity findings 
complement previous findings of insular involvement in 
multi-modal processing of stimuli [42] as well as source 
encoding [34,47]. 

Word-dependent source memory effects were also iden- 
tified in the bilateral parahippocampal cortices, extending 
to the vicinity of the right posterior hippocampus. These 
MTL effects reflect that activity of word-dependent source 
memory exhibits not only extensive object processing but 
also additional visuo-spatial and contextual processing 
for the words that were later remembered with correct 
source, compared with words without correct source, con- 
sistent with prior reports of parahippocampal activity for 
both spatial and nonspatial contexts [46,48]. On the other 
hand, there was no overlap between word-dependent 
source memory effects and the region where words were 
preferentially processed than pictures. Prior studies have 
shown overlap of content-dependent subsequent memory 
effects in the areas where corresponding content are pref- 
erentially processed [10,38,43]. The current, seemingly 
contradictory findings illustrate that successfully remem- 
bered words with accurate context are indeed the out- 
come of encoding in more of an 'object' manner due to 
the requirement of the study task. These findings suggest 
that adopting pictorial processing by engaging in the 
imagery-based encoding task was crucial for the formation 
of successful source memory only with words due to the 
interaction between the nature of study processing and 
the stimulus content. 13 In sum, the present findings dem- 
onstrate that the requirement of trial-general study pro- 
cessing influences the way that trial-specific stimulus 
content is experienced. Importantly, the present findings 
show that unitization is supported for different types of 
stimulus content. 

Conclusions 

We investigated neural activity specific to stimulus con- 
tent via integrated source encoding with words and pic- 
tures. As predicted, subsequent source memory effects 
were evident in the left lateral frontal and parietal cortex 
as well as the left perirhinal cortex extending to the hippo- 
campus, which is indicative of the significance of these 
regions in source memory formation for integrated object- 
context. Content-dependent source memory effects were 
found only with words in the ventral visual stream and the 
parahippocampal cortex, which reflects more visuo-spatial 
processing and contextual binding with words for suc- 
cessful source memory due to the requirement of imag- 
ery encoding operation. Collectively, the present findings 



demonstrate the effect of trial-general study processing in 
subsequent source memory effects depending on stimulus 
content through unitization. 

Methods 

Subjects 

Twenty-four subjects participated in the experiment (16 fe- 
males; 18-29 years). They were recruited from the Univer- 
sity of Texas-Arlington community and compensated for 
their participation. All subjects reported being right- 
handed, being native English speakers, and having no his- 
tory of neurological disease. Informed consent was obtained 
prior to participation. The experiment was approved by the 
University of Texas-Arlington and University of Texas 
Southwestern Medical Center Institutional Review Boards. 

Materials 

The experimental stimulus pool consisted of 175 con- 
crete words (4-9 letter nouns) and 175 line-drawings 
depicting nameable objects. Three-hundred critical items 
were selected from the stimulus pool to construct the 
study and test lists. For each subject, a study list of 200 
critical items (100 words and 100 line-drawing pictures) 
was created from the pool by random selection of items 
without replacement. Critical items were pseudo-randomly 
assigned to one of four color backgrounds (blue, green, 
red, or yellow), 25 words, and 25 pictures for each color. 
All words were presented in black, all pictures were pre- 
sented in black-white line-drawing form, and both words 
and pictures were overlaid on the colored background. 
Each study list comprised an intermixed presentation of 
critical items with 100 null trials interspersed among the 
experimental trials. A test list was consisted 200 studied 
items intermixed with 100 new items (50 words and 50 
pictures). Test items were presented on a gray background. 
Both study and test list sequences were constrained by 
no more than four consecutive presentations of a type of 
content or background color. An additional 50 items, 25 
words, and 25 pictures were used for practice trials. The 
composition and ordering of the study and test lists were 
made anew for each subject. 

All items were projected onto a screen viewed by sub- 
jects via a mirror mounted on the scanner head. For the 
study list, critical items were presented on one of four 
color backgrounds, while null events were presented on a 
gray background. For the test list, all items were presented 
on the gray background. All items were presented at a 
maximum visual angle of 7.1° x 7.1°. 

Procedure 

Subjects were given instructions and practice for experi- 
mental tasks prior to the experiment proper. The experi- 
ment consisted of two study-test cycles. For each study 
phase, a fixation cross was presented for 200 ms as a 
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warning signal for the upcoming item, followed by a 
study item superimposed on the colored background for 
2000 ms. The study item was replaced by a response 
prompt ('+') for 1300 ms prior to the onset of the next 
trial. Stimulus onset asynchrony was 3500 ms. Subjects 
were instructed to form an image of the object (e.g., 
'UMBRELLA') in the background color (e.g., 'RED') and 
to make a pleasantness judgment for the colored object 
('RED UMBRELLA'). Note that the pleasantness judgment 
was made based on the mental image that integrated ob- 
ject and color. This study instruction was aimed to facili- 
tate unitization of the object and color context. Subjects 
indicated their judgments by pressing a button with the 
index or middle finger of their right hand. The finger as- 
signment to each response was counterbalanced across 
subjects. Four colors (blue, green, red, or yellow) were 
used for backgrounds, and 50 items (half words and half 
pictures) were presented in each colored background. For 
null trials, a fixation character ('+') was continuously 
displayed for 3500 ms on a gray background, and no re- 
sponse was required. 

The test phase followed approximately 5 min after the 
end of the study phase. Test items were presented on 
the gray background. The content of the stimulus was 
kept consistent between study and test (e.g., if a stimulus 
was studied in a word form, it was also tested in the 
word form). Subjects were asked to judge whether the 
test item was previously studied in the experiment and, 
if it had been studied, to indicate in which color it was 
presented; subjects responded using a one-step source 
recognition response. Each study and test session lasted 
about 20 minutes. 

fMRI scanning 

A Philips Achieva 3T MR scanner (Philips Medical Sys- 
tems, Andover, MA) fitted with a 32-channel RF receiver 
head coil was used to acquire both Ti-weighted anatomical 
volume images (256 x 238 matrix, 1 mm 3 voxels) and T 2 - 
weighted echo-planar images (EPIs) (80 x 80 matrix, 3 mm x 
3 mm in-plane resolution, axial acquisition, flip angle 70°, 
TE 30 ms) per volume. Each EPI volume comprised thirty 
3 mm-thick axial slices acquired in a descending sequential 
order and separated by 1 mm, providing coverage of almost 
the entire brain. Data were acquired during the study and 
test phases in four scanning sessions comprising 270 vol- 
umes each, with a repetition time (TR) of 2 s. Five ad- 
ditional volumes were collected at the beginning of each 
run, but these were discarded to allow for T x equilibration. 
The 3.5 s SOA allowed for an effective sampling rate of the 
hemodynamic response of 2 Hz. 

fMRI data analysis 

Data preprocessing and statistical analyses were performed 
with Statistical Parametric Mapping (SPM 8, Wellcome 



Department of Cognitive Neurology, London, UK: http:// 
www.fil.ion.ucl.ac.uk), implemented in MATLAB 9 (Math- 
works, Natick, MA). For each subject, functional images 
were registered to the first image of each scan session and 
temporally realigned to the middle slice using sine inter- 
polation. The resulting data were normalized to a standard 
EPI template based on the Montreal Neurological Institute 
(MNI) reference brain and resampled into 3 mm isotropic 
voxels using nonlinear basis functions [49]. The normal- 
ized images were smoothed with an isotropic 8 mm full- 
width half-maximum Gaussian kernel. The time series in 
each voxel were high-pass filtered to 1/128 Hz to remove 
low-frequency noise and scaled to a grand mean of 100 
across both voxels and scans. Ti-weighted anatomical im- 
ages were coregistered to the mean EPI volume and nor- 
malized to a standard Tj template of the MNI brain. After 
normalization, an across-subject mean anatomical image 
was created as the study-specific template for identifying 
brain regions. 

Statistical analysis was performed on the study phase 
data using a two-stage mixed effects model for investigat- 
ing encoding activity associated with content-dependent 
source memory effects through unitization of the item 
and context. In the first stage, neural activity was modeled 
by delta functions at stimulus onset. The event-related 
blood oxygen-level dependent (BOLD) response was mod- 
eled by convolving these neural functions with a canonical 
hemodynamic response function (HRF) and its temporal 
and dispersion derivatives. In addition, six regressors were 
modeled for movement-related variance, and session- 
specific constant terms were employed to model the mean 
image intensity in each session. Parameter estimates for 
events of interest were measured for each subject using 
a General Linear Model. Non-sphericity of the error co- 
variance was accommodated by an AR(i) model in which 
the temporal autocorrelation was estimated by pooling 
over suprathreshold voxels [50]. The parameters for each 
covariate and the hyperparameters governing the error co- 
variance were estimated using Restricted Maximum Like- 
lihood (ReML). In the second stage, linear contrasts of 
these subject-specific parameter estimates were computed, 
treating subjects as a random effect. 

For the analysis of subsequent source memory effects by 
stimulus content, four events of interest were defined: 
'word-source correct' (studied words that were recognized 
with correct study color); 'word-source incorrect' (studied 
words that were recognized albeit with incorrect study 
color); 'picture-source correct' (studied pictures that were 
accurately judged with study color); and 'picture-source 
incorrect' (studied pictures that were responded with in- 
correct color). All other study trials including item misses 
(studied items that were incorrectly judged as new) and 
no responses were modeled as events of no-interest, due 
to an insufficient number of trials. c 
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At the whole brain level, only effects surviving a voxel- 
wise threshold of p < .001 and a corrected cluster-wise 
threshold of p < .05 (greater than 16 voxels) were inter- 
preted. When exclusive masking was applied to identify 
voxels where effects were not shared between two con- 
trasts, the mask threshold was set at a two-tailed threshold 
of p < .1. Note that the more liberal the threshold of an 
exclusive mask, the more conservative the masking pro- 
cedure. The threshold for the inclusive mask was set to 
p < .01. The peak voxels of clusters exhibiting reliable ef- 
fects are reported in MNI coordinates. For the regions 
identified from main analyses, region-specific parameter 
estimates were extracted by peak voxel activity of a cluster 
for each subject and subjected to group-level statistical 
tests. The significance level for the region-specific param- 
eter estimates was set top < .05. 

Endnotes 

a We thank Reviewer for clarification of this inter- 
pretation. 

b At the MTL level, subsequent source memory effects 
dependent on words may be accounted by recollection. 
That is, words that were recognized with correct source 
information recruited more activity related with recollec- 
tion process in the parahippocampal cortex and the 
hippocampus due to binding of the item and context, 
compared with words without source memory. To put 
it differently, source correct words were more likely 
the words that were accompanied with the recollection 
process of binding the item and context compared with 
source incorrect words, whereas subsequent source mem- 
ory dependent on pictures was not significantiy distin- 
guished by recollection process. We thank Reviewer for 
this suggestion. 

c Only 7 out of 24 participants had sufficient numbers 
of trials (> 10) for both words and pictures. 
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